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This  report  for  the  research  grant  F49620-92-J-0223,  titled  Development  of  function-based  mod¬ 
eling  for  generic  object  recognition,  covers  the  period  1  August  1994  through  31  July  1995.  This 
report  provides  a  summary  of  the  major  areas  of  activity  and  the  current  results  in  each  area,  a 
list  of  the  Ph.D.  candidates  working  on  the  different  areas  of  the  project  and  a  list  of  publica¬ 
tions.  Copies  of  recent  publications  from  Transactions  on  PAMI  and  the  1995  IEEE  Symposium 
on  Computer  Vision  are  attached. 

1  Major  areas  of  activity 

1.1  Use  of  partial  shape  models 

Complete  shape  information  about  an  unknown  object  is  generally  not  available  in  an  observer’s 
first  view  of  the  object.  For  this  reason,  we  have  been  working  on  the  extraction  of  partial  shape 
models  from  an  initial  view  of  an  object.  We  have  given  the  name  “OPUS”  (Object  Plus  Unseen 
Space)  to  the  type  of  shape  model  that  we  construct  from  an  initial  view  of  a  3-D  object.  The 
results  of  our  initial  work  on  function-based  recognition  from  OPUS  shape  models  have  been 
evaluated  using  a  database  of  over  200  real  laser  range  finder  images.  Out  of  concerns  for 
generality  and  robustness,  we  are  evaluating  the  algorithm  for  OPUS  construction  with  range 
data  from  a  structured  light  scanner  as  well  as  data  from  the  time-of-flight  laser  range  finder. 
This  line  of  work  is  being  extended  to  include  incremental  construction  of  OPUS  models  from 
multiple  views  and  the  development  of  function-driven  strategies  for  selection  of  a  sequence  of 
views.  Portions  of  this  work  have  been  accepted  for  publication  in  Transactions  on  Pattern 
Analysis  and  Machine  Intelligence. 

1.2  Application  of  machine  learning  techniques 

The  application  of  machine  learning  techniques  to  computer  vision  systems  has  recently  begun 
to  receive  increased  attention  in  the  community.  In  the  GRUFF  project,  we  have  been  looking 
at  possible  ways  to  simplify  the  problem  of  extending  the  domain  of  competence  to  include 
new  object  categories.  At  the  moment,  this  involves  a  fair  amount  of  handcrafting  of  various 
parameters  for  the  knowledge  base  that  describes  the  function  appropriate  to  the  object  category. 
In  particular,  for  each  knowledge  primitive  invocation  in  the  system,  there  can  be  four  values 
which  defined  the  evaluation  measure  for  the  invocation.  (In  other  words,  at  each  place  that 
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an  observation  about  width,  depth,  area,  relative  orientation  or  such  is  made,  there  are  four 
values  to  be  given  to  specify  the  range  of  appropriate  values.)  We  have  developed  a  machine 
learning  technique  to  automatically  derive  an  appropriate  set  of  values  from  a  small  set  of  labeled 
examples.  This  work  has  been  accepted  for  publication  in  the  Journal  of  Artificial  Intelligence 
Research. 

1.3  Extensions  to  the  domain  of  articulated  objects 

We  are  working  on  the  extension  of  the  domain  of  competence  of  the  GRUFF  system  to  include 
objects  composed  of  rigid  parts  with  articulated  connections  between  them.  WTile  the  current 
version  of  GRUFF  can  handle  objects  such  as  “hammer”  and  “screwdriver”,  we  are  aiming  at 
a  version  which  would  also  handle  objects  such  as  “pliers” ,  “scissors” ,  “adjustable  wrench”  and 
“hand  drill” .  A  description  of  the  early  results  of  this  work  has  been  accepted  for  publication 
in  Computer  Vision  and  Image  Understanding  (formerly  titled  CVGIP:  Image  Understanding. 
Implementation  and  evaluation  of  this  system  is  continuing. 

1.4  General  publications 

In  addition  to  the  above-listed  research  topics  and  publications,  Louise  Stark  has  served  as  guest 
editor  of  a  special  issue  of  Computer  Vision  and  Image  Understanding ,  tentatively  scheduled 
for  September  1995  publication,  devoted  to  the  role  of  functionality  in  object  recognition  and 
Professors  Stark  and  Bowyer  are  preparing  a  monograph  on  the  Gruff  project  to  be  publi,shed 
by  W^rld  Scientific  Publishing  Company. 
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Ph.D.  candidates  working  on  the  project 

2.1  Melanie  Sutton 

Ms.  Melanie  Sutton  in  currently  in  her  second  year  in  the  graduate  program  at  USF.  She  has 
contributed  to  several  stages  of  the  generalization  of  the  basic  GRUFF  system.  She  was  the 
primary  contributor  in  adding  the  example  domain  of  the  superordinate  categor\^  “dishes”  to 
the  system.  She  has  also  been  collaborating  on  other  areas  of  the  project. 

2.2  Adam  Hoover 

Mr.  Adam  Hoover  is  currently  in  his  second  year  in  the  graduate  program  at  USF.  He  has  been 
working  primarily  in  the  area  of  extending  the  GRUFF  system  to  reason  about  input  shape 
descriptions  of  the  type  acquired  from  a  real  mobile  robot  and  laser  range  finder.  He  has  also 
been  collaborating  on  other  areas  of  the  project. 

2.3  Kevin  Green 

Mr.  Kevin  Green  is  currently  in  his  fourth  year  in  the  graduate  program  at  USF.  He  has  been 
working  out  the  conceptual  extensions  necessary  to  extend  the  domain  of  objects  that  can  be 
handled  to  include  articulated  objects;  that  is.  objects  with  moving  parts.  Simple  examples 
would  be  scissors,  pliers  and  adjustable  wrenches. 
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3  List  of  Publications  For*  This  Year  of  the  Project 

•  K.  Woods,  D.  Cook,  L.  Hall,  L.  Stark,  and  K.  Bowyer.  Learning  Membership  Functions  in 
a  Function-Based  Object  Recognition  System  accepted  to  appear  in  Journal  of  Artificial 
Intelligence  Research. 

•  Hoover,  A.W.,  Goldgof,  D.B.  and  Bowyer,  KW*.  Extracting  a  Valid  Boundary  Representa¬ 
tion  from  a  Segmented  Range  Image,  accepted  to  appear  in  IEEE  Transactions  on  Pattern 
Analysis  and  Machine  Intelligence. 

•  Green,  K.,  Eggert,  D.,  Stark,  L.  and  Bowyer,  K.W.  Generic  recognition  of  articulated  ob¬ 
jects  through  reasoning  about  potential  function,  accepted  to  appear  in  Computer  Vision 
and  Image  Understanding . 

•  Sutton,  M.,  Stark,  L.  and  Bowyer,  K.  GRUFF-3:  Generalizing  the  domain  of  a  function- 
based  recognition  system.  Pattern  Recognition  27  (12),  1743-1766,  December  1994. 

•  Stark,  L.,  Bowyer,  ICW.,  Woods,  K..  Hall,  L.,  and  Cook,  D.,  Application  of  learning 
techniques  in  a  function-based  recognition  system,  to  appear  in  Symbolic  Visual  Learning 
K.  Ikeuchi  and  M  Veloso,  editors,  Oxford  University  Press,  1995. 

•  Hoover,  A.,  Jean-Baptiste,  G.,  Jiang.  X.,  Flynn,  P.J.,  Bunke,  H.  and  Bowyer,  K.  Range  im¬ 
age  segmentation:  the  user’s  dilemma,  1995  IEEE  International  Symposium  on  Computer 
Vision,  Miami,  Florida  (November  1995). 

•  Sanocki,  T.,  Bowyer,  K.  Adair,  J.,  and  Sarkar,  S.  Are  real  edges  sufficient  for  object 
recognition?,  Full  version  submitted  for  journal  review,  early  version  presented  at  Annual 
Meeting  of  Association  for  Research  in  Vision  and  Opthamology,  Palm  Beach,  Florida 
(May  1995). 
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